3D场景由大量背景点主导，这对于主要需要集中在前景对象的检测任务是多余的。在本文中，我们分析了现有的稀疏3D CNN的主要组成部分，发现3D CNN忽略了数据的冗余，并在下降过程中进一步扩大了数据，这带来了大量的多余和不必要的计算间开销。受到这一点的启发，我们提出了一个名为“空间修剪稀疏卷积”（SPS-CONV）的新型卷积操作员，其中包括两个变体，空间修剪的Submanifold稀疏卷积（SPSS-CONV）和空间修剪的常规稀疏卷积（SPRS-CONV），包括这是基于动态确定冗余降低关键领域的想法。我们验证该幅度可以作为确定摆脱基于学习方法的额外计算的关键领域的重要提示。提出的模块可以轻松地将其纳入现有的稀疏3D CNN中，而无需额外的架构修改。关于Kitti，Waymo和Nuscenes数据集的广泛实验表明，我们的方法可以在不损害性能的情况下实现超过50％的GFLOPS。

在本文中，我们提出了广义参数对比度学习（GPACO/PACO），该学习在不平衡和平衡数据上都很好地工作。基于理论分析，我们观察到，受监督的对比损失倾向于偏向高频类别，从而增加了学习不平衡的学习难度。我们从优化的角度介绍了一组参数班的可学习中心，以重新平衡。此外，我们在平衡的环境下分析了GPACO/PACO损失。我们的分析表明，GPACO/PACO可以适应地增强同一等级样品的强度，因为将更多的样品与相应的中心一起拉在一起并有益于艰难的示例学习。长尾基准测试的实验表明了长尾识别的新最先进。在完整的Imagenet上，与MAE模型相比，从CNN到接受GPACO损失训练的视觉变压器的模型显示出更好的泛化性能和更强的鲁棒性。此外，GPACO可以应用于语义分割任务，并在4个最受欢迎的基准测试中观察到明显的改进。我们的代码可在https://github.com/dvlab-research/parametric-contrastive-learning上找到。

在过去的几年中，用于计算机视觉的深度学习技术的快速发展极大地促进了医学图像细分的性能（Mediseg）。但是，最近的梅赛格出版物通常集中于主要贡献的演示（例如，网络体系结构，培训策略和损失功能），同时不知不觉地忽略了一些边缘实施细节（也称为“技巧”），导致了潜在的问题，导致了潜在的问题。不公平的实验结果比较。在本文中，我们为不同的模型实施阶段（即，预培训模型，数据预处理，数据增强，模型实施，模型推断和结果后处理）收集了一系列Mediseg技巧，并在实验中探索了有效性这些技巧在一致的基线模型上。与仅关注分割模型的优点和限制分析的纸驱动调查相比，我们的工作提供了大量的可靠实验，并且在技术上更可操作。通过对代表性2D和3D医疗图像数据集的广泛实验结果，我们明确阐明了这些技巧的效果。此外，根据调查的技巧，我们还开源了一个强大的梅德西格存储库，其每个组件都具有插件的优势。我们认为，这项里程碑的工作不仅完成了对最先进的Mediseg方法的全面和互补的调查，而且还提供了解决未来医学图像处理挑战的实用指南，包括但不限于小型数据集学习，课程不平衡学习，多模式学习和领域适应。该代码已在以下网址发布：https：//github.com/hust-linyi/mediseg

在语义细分中进行了无监督的域的适应，以减轻对昂贵像素的依赖的依赖。它利用标有标记的源域数据集以及未标记的目标域图像来学习分割网络。在本文中，我们观察到现有的域不变学习框架的两个主要问题。 （1）由于特征分布对齐而分心，网络不能专注于分割任务。 （2）拟合源域数据很好地损害了目标域性能。为了解决这些问题，我们提出了减轻过度拟合源域的脱钩，并使最终模型能够更多地专注于细分任务。此外，我们提出自我歧视（SD），并引入辅助分类器，以使用伪标签学习更多歧视目标域特征。最后，我们建议在线增强自我训练（OEST），以在线方式上下文提高伪标签的质量。实验表明，我们的方法优于现有的最新方法，广泛的消融研究验证了每个组件的有效性。代码可在https://github.com/dvlab-research/decouplenet上找到。

培训语义分割模型需要大量的精细注释数据，使得很难快速适应不满足这种情况的新型类。很少拍摄的分割（FS-SEG）用许多约束来解决这个问题。在本文中，我们介绍了一种新的基准，称为广义的少量语义分割（GFS-SEG），分析了同时分割了具有很少的例子和基本类别的新型类别的泛化能力。第一研究表明，以前的代表性最先进的FS-SEG方法在GFS-SEG中缺乏，并且性能差异主要来自FS-SEG的约束设置。为了制作GFS-SEG易旧的，我们设置了GFS-SEG基线，可以在原始模型上实现不良性能的体现性能。因此，由于上下文对于语义分割是必不可少的，我们提出了显着提高性能的上下文感知原型学习（CAPL）1）利用支持样本的共同发生，以及2）将上下文信息动态地丰富到分类器，对每个查询映像的内容进行调节。两项贡献都是通过实验证明具有实际实际优点的贡献。对Pascal-VOC和Coco的广泛实验表现出CAPL的有效性，CAPL通过实现竞争性能来概括为FS-SEG。代码将公开可用。

Given the increasingly intricate forms of partial differential equations (PDEs) in physics and related fields, computationally solving PDEs without analytic solutions inevitably suffers from the trade-off between accuracy and efficiency. Recent advances in neural operators, a kind of mesh-independent neural-network-based PDE solvers, have suggested the dawn of overcoming this challenge. In this emerging direction, Koopman neural operator (KNO) is a representative demonstration and outperforms other state-of-the-art alternatives in terms of accuracy and efficiency. Here we present KoopmanLab, a self-contained and user-friendly PyTorch module of the Koopman neural operator family for solving partial differential equations. Beyond the original version of KNO, we develop multiple new variants of KNO based on different neural network architectures to improve the general applicability of our module. These variants are validated by mesh-independent and long-term prediction experiments implemented on representative PDEs (e.g., the Navier-Stokes equation and the Bateman-Burgers equation) and ERA5 (i.e., one of the largest high-resolution data sets of global-scale climate fields). These demonstrations suggest the potential of KoopmanLab to be considered in diverse applications of partial differential equations.

Humans have internal models of robots (like their physical capabilities), the world (like what will happen next), and their tasks (like a preferred goal). However, human internal models are not always perfect: for example, it is easy to underestimate a robot's inertia. Nevertheless, these models change and improve over time as humans gather more experience. Interestingly, robot actions influence what this experience is, and therefore influence how people's internal models change. In this work we take a step towards enabling robots to understand the influence they have, leverage it to better assist people, and help human models more quickly align with reality. Our key idea is to model the human's learning as a nonlinear dynamical system which evolves the human's internal model given new observations. We formulate a novel optimization problem to infer the human's learning dynamics from demonstrations that naturally exhibit human learning. We then formalize how robots can influence human learning by embedding the human's learning dynamics model into the robot planning problem. Although our formulations provide concrete problem statements, they are intractable to solve in full generality. We contribute an approximation that sacrifices the complexity of the human internal models we can represent, but enables robots to learn the nonlinear dynamics of these internal models. We evaluate our inference and planning methods in a suite of simulated environments and an in-person user study, where a 7DOF robotic arm teaches participants to be better teleoperators. While influencing human learning remains an open problem, our results demonstrate that this influence is possible and can be helpful in real human-robot interaction.

We introduce a new tool for stochastic convex optimization (SCO): a Reweighted Stochastic Query (ReSQue) estimator for the gradient of a function convolved with a (Gaussian) probability density. Combining ReSQue with recent advances in ball oracle acceleration [CJJJLST20, ACJJS21], we develop algorithms achieving state-of-the-art complexities for SCO in parallel and private settings. For a SCO objective constrained to the unit ball in $\mathbb{R}^d$, we obtain the following results (up to polylogarithmic factors). We give a parallel algorithm obtaining optimization error $\epsilon_{\text{opt}}$ with $d^{1/3}\epsilon_{\text{opt}}^{-2/3}$ gradient oracle query depth and $d^{1/3}\epsilon_{\text{opt}}^{-2/3} + \epsilon_{\text{opt}}^{-2}$ gradient queries in total, assuming access to a bounded-variance stochastic gradient estimator. For $\epsilon_{\text{opt}} \in [d^{-1}, d^{-1/4}]$, our algorithm matches the state-of-the-art oracle depth of [BJLLS19] while maintaining the optimal total work of stochastic gradient descent. We give an $(\epsilon_{\text{dp}}, \delta)$-differentially private algorithm which, given $n$ samples of Lipschitz loss functions, obtains near-optimal optimization error and makes $\min(n, n^2\epsilon_{\text{dp}}^2 d^{-1}) + \min(n^{4/3}\epsilon_{\text{dp}}^{1/3}, (nd)^{2/3}\epsilon_{\text{dp}}^{-1})$ queries to the gradients of these functions. In the regime $d \le n \epsilon_{\text{dp}}^{2}$, where privacy comes at no cost in terms of the optimal loss up to constants, our algorithm uses $n + (nd)^{2/3}\epsilon_{\text{dp}}^{-1}$ queries and improves recent advancements of [KLL21, AFKT21]. In the moderately low-dimensional setting $d \le \sqrt n \epsilon_{\text{dp}}^{3/2}$, our query complexity is near-linear.

We study the task of learning state representations from potentially high-dimensional observations, with the goal of controlling an unknown partially observable system. We pursue a direct latent model learning approach, where a dynamic model in some latent state space is learned by predicting quantities directly related to planning (e.g., costs) without reconstructing the observations. In particular, we focus on an intuitive cost-driven state representation learning method for solving Linear Quadratic Gaussian (LQG) control, one of the most fundamental partially observable control problems. As our main results, we establish finite-sample guarantees of finding a near-optimal state representation function and a near-optimal controller using the directly learned latent model. To the best of our knowledge, despite various empirical successes, prior to this work it was unclear if such a cost-driven latent model learner enjoys finite-sample guarantees. Our work underscores the value of predicting multi-step costs, an idea that is key to our theory, and notably also an idea that is known to be empirically valuable for learning state representations.

Video semantic segmentation (VSS) is beneficial for dealing with dynamic scenes due to the continuous property of the real-world environment. On the one hand, some methods alleviate the predicted inconsistent problem between continuous frames. On the other hand, other methods employ the previous frame as the prior information to assist in segmenting the current frame. Although the previous methods achieve superior performances on the independent and identically distributed (i.i.d) data, they can not generalize well on other unseen domains. Thus, we explore a new task, the video generalizable semantic segmentation (VGSS) task that considers both continuous frames and domain generalization. In this paper, we propose a class-wise non-salient region generalized (CNSG) framework for the VGSS task. Concretely, we first define the class-wise non-salient feature, which describes features of the class-wise non-salient region that carry more generalizable information. Then, we propose a class-wise non-salient feature reasoning strategy to select and enhance the most generalized channels adaptively. Finally, we propose an inter-frame non-salient centroid alignment loss to alleviate the predicted inconsistent problem in the VGSS task. We also extend our video-based framework to the image-based generalizable semantic segmentation (IGSS) task. Experiments demonstrate that our CNSG framework yields significant improvement in the VGSS and IGSS tasks.